Tom W professional judgment experiment

The Tom W professional judgment experiment is a central demonstration of the representativeness heuristic and one of the clearest cases in which an easier question displaces the harder one. Participants were asked to estimate the probability that Tom W belonged to one of nine graduate specialties, but their answers closely followed the perceived similarity between Tom’s personality and the stereotype of each field. The experiment showed that a vivid individual description can overpower base-rate information, even when the reasoner knows that base rates matter and understands that the evidence is unreliable.

The Initial Probability Task

The experiment began with a simple ranking problem. Tom W was described as a graduate student at the main university in the respondent’s state, and participants had to rank business administration, computer science, engineering, humanities and education, law, medicine, library science, physical and life sciences, and social science and social work according to the likelihood that Tom studied in each field. The natural statistical solution was to use the base rates: if Tom had been selected at random from all graduate students, the specialties with larger enrollments should receive higher probability in the absence of other information.

This first task made the relevant logic explicit. The proportion of students belonging to a field is its base rate, just as the proportion of red or green marbles in an urn determines the likelihood of drawing either color. A student selected without distinguishing information should therefore be more likely to come from a large field such as humanities and education than from a small field such as computer science or library science. Base-rate reasoning was not presented as an advanced technical maneuver; it was the obvious response to a random-selection problem with no individual evidence.

The Personality Sketch and Stereotypes

The second task changed the informational setting by supplying a personality description. A psychologist had written the sketch during Tom’s senior year in high school on the basis of psychological tests of uncertain validity. Tom was described as highly intelligent but lacking true creativity, driven by order, clarity, competence, and tidy systems. His writing was dull and mechanical, though occasionally enlivened by corny puns and science-fiction flashes of imagination. He supposedly had little sympathy for others, disliked social interaction, was self-centered, and nevertheless possessed a deep moral sense.

Participants were then asked to rank the same nine specialties according to how similar Tom was to the typical graduate student in each field. This question was easier than the probability question because it required the retrieval or construction of stereotypes rather than the integration of statistical information with uncertain personal evidence. When the experiment was first conducted in the early 1970s, the average ordering began with computer science and engineering. The sketch was designed to fit the image of students in small, technically oriented groups, especially computer scientists, librarians, and engineers, while fitting the much larger fields of humanities and education and social science and social work poorly.

Tom was therefore an intentionally anti-base-rate character. His description pulled in one direction through representativeness, while the enrollment figures pulled in the opposite direction through statistical prevalence. The contrast was not accidental decoration: it created a direct test of whether respondents would preserve the distinction between similarity and probability when both were made psychologically available.

Prediction and Substitution

The decisive task asked graduate students in psychology to rank the nine fields by the likelihood that Tom was studying in each one. These participants knew the relevant statistical facts. They were familiar with base rates, and they knew that the personality sketch came from tests whose validity was uncertain. The researchers nevertheless expected them to focus almost entirely on similarity and to rank computer science highly because Tom’s description received the strongest representativeness score there.

The experiment grew out of a deliberately constructed description. During a year spent in Eugene, Daniel Kahneman and Amos Tversky worked extensively on the problem, and Daniel Kahneman composed Tom’s sketch during an overnight session. The first colleague to arrive the next morning was Robyn Dawes, a sophisticated statistician and a skeptic of intuitive judgment. When asked to guess Tom’s profession, Robyn Dawes tentatively answered, computer scientist. The episode was striking precisely because he understood the relevance of base rates; once it was mentioned, he immediately recognized that he had neglected them. His spontaneous response had nevertheless substituted representativeness for the probability he had been asked to assess.

The larger study collected answers from 114 psychology graduate students at three major universities, all of whom had taken several statistics courses. Their probability rankings did not differ from their similarity rankings. The substitution was effectively complete: the participants behaved as though the probability question had asked which field best matched Tom’s stereotype. Statistical training did not prevent the error from occurring automatically, because the easier similarity judgment supplied a coherent answer before the more demanding probabilistic reasoning was carried out.

Why the Error Matters

Similarity and probability obey different logical constraints. A judgment of similarity may properly ignore the size of the categories and may remain unchanged even when the description is inaccurate. Probability judgments cannot ignore either consideration. A small specialty can contain people who strongly resemble a stereotype while still being less likely than a large specialty for a randomly selected student. Likewise, a personality sketch based on uncertain tests cannot be treated as decisive evidence merely because it produces a vivid and coherent impression.

The experiment illustrates question substitution. The requested question was difficult: what is the probability that Tom studies in each field, given the base rates and the limited reliability of the description? The substituted question was easier: how closely does Tom resemble the stereotypical student in each field? Because the answer to the substitute question felt informative and psychologically satisfying, it was accepted as an answer to the original one. The resulting confidence did not come from a valid combination of evidence; it came from the fluency of the stereotype.

Wider Significance for Prediction

The Tom W experiment belongs to a broader argument about the limits of expert judgment. Philip Tetlock’s study of 284 people who earned their living commenting on political and economic trends gathered more than 80,000 probability forecasts and found that the experts performed worse than an equal-probability baseline across three possible outcomes. Even in areas they knew best, they were not significantly better than nonspecialists, while greater knowledge could intensify the illusion of skill and overconfidence.

Paul Meehl’s comparison of clinical predictions with simple statistical formulas reached a related conclusion. In studies of outcomes such as freshman grades, parole violations, pilot-training success, and criminal recidivism, formulas matched or exceeded trained professionals; in a later body of roughly two hundred comparisons, algorithms performed significantly better in about 60 percent, with the remaining studies generally ending in a draw. The advantage of a tie belonged to the formula because fixed rules were normally cheaper and more consistent than expert judgment.

The reason is not simply that experts lack information. Human judges are inconsistent when they combine complex evidence, sometimes giving different evaluations to the same case only minutes apart. Experienced radiologists contradicted their own classifications of the same chest X-ray about 20 percent of the time, and similar inconsistency appeared among auditors and other professionals. Formulas return the same result when given the same inputs, whereas context-dependent System 1 can shift with unnoticed environmental influences.

The practical lesson is not that judgment should never be used. A decisive and rare fact can properly override a formula, as in the broken-leg rule: information that a person has broken a leg today may defeat a routine prediction that the person will go to the movies. But absent such a decisive exception, the Tom W pattern warns against allowing a polished individual story to erase category size, evidence quality, and the difference between resemblance and probability. It is an experiment about a single graduate student only on the surface; underneath, it is a test of whether intuitive coherence can be mistaken for statistical truth.